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Abstract— Blind humans are confronted with numerous difficulties on the subject of spending their regular lives. This 
research study investigates the efficacy of the diverse systems developed for the navigational assistance of the blind and 
visually impaired humans and proposes a new system. While traditional systems rely on the development of assisting 
hardware such as canes, suits and sticks, our proposed system explores the utilization of depth cameras. In contrast to the 
general cameras, the depth cameras can provide the distance of the object from the camera, in the field of view. This 
information in conjunction with image processing has been utilized in the current research work for efficient identification 
of obstacles and provision of accurate navigational directives. Currently, the proposed system can inform the user of any 
object(s) appearing in their frontal path and suggest directions to bypass the object. Validation of the system was 
performed in real time and it suggests the system having high potential for practical use. 
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Il. TECHNIQUES AND METHODS 


I. INTRODUCTION This section briefly reviews and summarizes approach 


In today era the number visually impaired people is and strategies of those sensors but especially focused 


increasing day by day. A lot of assistance is provided to may be on strategies and methods for object detection. 


them in various forms. Nonetheless, the detection of Systems and techniques which were critical in terms of 


obstacles is a great challenge for researcher and primary efficiency, robustness, cost and practicality were 


focus 1s on the utilization of various forms of sensors for selected for analysis. This included selection based on 


object detection in the close vicinity and then providing sensor type as well. Many sensors are available for 


feedback to the user in the form of directives to help sensing motive in literature. Some of those are used 


them navigate. detect simple warm bodies like human and animals. A 


few detectors are not appropriate for each sort of 
Many sensors are available for detection purpose in the , _ 

io environment like. an infrared radiation sensor, Itdoes not 
literature. Some of those sensors are extensively utilized 
work desirably in a dark environment. 
in domestic protection structures and are supposed to 


stumble on simplest heat bodies, including human A detecting component could be a gadget/module that 


beings and animals. Others are utilized in robots to keep identifies and answer a type of contribution from the 


away from boundaries and become aware of matters and physical setting. That information could likewise be 


impediments. Finally, positive sensors, including sound waves, IR waves, light and so on. There are 2 


methane fuel line detection sensors, are utilized for a verities of sensors, active and passive. The first are 


particular purpose. Infrared sensors, for example, are not active sensors that require energy to operate, such as 


suitable for each situation. It does now no longer provide infrared and ultrasonic sensors [1]. 


top results in a darkish environment. These are discussed 


in detail in Section 2. An emitting source and a collecting source are combined 


in the IR sensing component. After striking an item in 


Various systems have been developed for the front of the sensor, the transmitting source transmits IR 


navigational assistance. Section 2 critically reviews and signals, which are likewise received by the receiving 


summarizes the existing techniques and methods mainly source. To create and transmit IR signals, the IR 


focusing on object detection. Section 3 presents the 
proposed scheme based on a depth camera while Section 
4 demonstrates the experimental setup and results. 


Concluding remarks are presented in Section 5 including 
future directions. Section 2 below presents the current 
techniques and methods. 
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detecting device requires electricity. Passive sensors, 
such as the PIR sensing element [2], which detects heat 
radiation generated by hot bodies such as humans and 
animals, are the second class of sensors. 


For intelligent structures such as intelligent car parking 
systems, industrial intelligent robots, and blind aid 
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systems, a variety of sensors are commonly employed. 
In [1,] Meppurath et al. demonstrated a sensitive stick 
for visually impaired people that relies on ultrasonic 
sensors. By sensing the impediment in front of it, this 
stick may direct you to a safe path. This smart stick has 
a 4 mrange and vibrates to alert the user to any obstacles 
detected. 


In [2,] Sekar et al. established a facility to assist visually 
impaired persons in walking. In this ultrasonic, 
humidity, and temperature sensor company, three 
sensors are employed. The ultrasonic sensor is utilised 
in this installation to detect impediments ahead of vision 
impaired persons. 


This help system for the blind is dubbed "Navigation aid 
for the visually impaired (NAVI)" by Saiarayan et al. in 


[1]. 


Stereo headphones, a single board computing system, 
and a vision sensor mounted on the helmet make up this 
system. The image is captured using a vision sensor, and 
then analysed using their algorithm to locate an 
impediment in the image. 


In [2,] Ismail et al. developed a robot that identifies 
obstructions in its route and continues along it without 
them. The infrared sensor and the passive infrared 
sensor are both used by this robot. 


These sensors keep an eye on the robot's front end and 
train it to follow an obstacle-free path. The programme 
iS written in C++, and the Arduino microcontroller is 
programmed using the Arduino IDE 1.6.5 software. 





Figure I: Obstacle Avoiding Robot [1] 


As shown in Figure 2, Bagwan M et al. created a robot 
for the purpose of picking up, positioning and 
monitoring industrial personnel. 


Infrared and passive infrared sensors are used to detect 
possible thieves. IR sensors work during the day, while 
PIR sensors work at night. 
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Figure 2: Industrial Robot [2] 


Praveen and Roy in their research work focused on the 
estimation of depth from a single image for blind 
navigation system [3]. For obstacle extraction from the 
foreground, they worked on firstly detecting object 
edges in the image, joining the edges which were 
followed by noise removal. Ankit et al. mainly focused 
on the navigation system for the visually impaired 
people using a portable stick having GPS, Camera, and 
an ultrasonic sensor. The ultrasonic sensor can detect the 
obstacles within.arange of 180 degrees...After detection, 
it alerts the person by the help of a buzzer or vibrator [4]. 


Wahab et al. in their research work discussed developed 
a smart cane by the help of a number of ultrasonic 
sensors and servo motors [5]. The ultrasonic sensor 
helps out to detect the barrier in front of visually 
impaired person by warning him through voice or 
buzzer, as shownin Figure 3. They, alsovachieved the 
target of sensing the water at 0.5 cm deep. It has some 
limitations like smart cane can’t be folded and also it is 
expensive to buy for a low income person. 





Figure 3: The Smart Cane [5] 


Furthermore, another paper shows the methodology of 
navigation system using infrared ways develop by 
Amyed et al. [6]. They developed the system using 
infrared microcontroller to detect the obstacle as well as 
the shape and material of obstacle, shown in Figure 4. 
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The person can instruct by using speaker which are 
mounted on the hat of person as shown below. 





Figure 4: Infrared microcontroller based navigation 


assistance System. 


Ayat et al. they develop the cheap navigation system for 
visually impaired people. Assistive infrared sensors are 
used into smart stick to detect the object coming in front 
of a person with in arrange of 2m [7].in their research 
work they produced six different voices when stick 
detect the obstacle. Bagwan et al. in [8] designed a robot 
with the purpose of select and vicinity .and surveillance 
of enterprise employees as appeared in parent 2. IR and 
PIR sensor right here are applied to become aware of 
any thief. IR sensor work at day time while PIR sensors 
work during the night time. 


Ali Khan et al. in their research work utilized ultrasonic 
sensors for detecting objects in close proximity [9]. 
Figure 5 shows their developed system. The sensors 
were placed on the arms and around the knees and one 
on top of the head. This way objects to the right and left, 
below belt level and in the front could be detected and 
buzzer would be activated when the ultrasonic sensor 
would detect any object. 





wr me |, 
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Figure 5: Ultrasonic sensors placed on various parts 
of the body for object detection [9]. 
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Section 3 presents the proposed system based on depth 
camera. 


II. PROPOSED DEPTH BASED NAVIGATION 
SYSTEM 
The proposed scheme is based on depth camera. An 


overview of the proposed scheme is shown in Figure 6. 
Depth image 
input 


Separate Depth 
Channel 





Segment regions on 
the basis of depth 


Provide decision on 
obstacle in path 


Inform user by 
audio 


Figure 6: Overview of the proposed method. 


The steps are laid.out as follows: 

e First the depth camera captures the image. 

e The depth channel is separated and input to the 
segmentation algorithm. 

e The segmentation stage segments the depth for 
obstacle distances. 

e Number of objects and their relevant distances 
are informed to the user through audio 
messages for their decision making. 


IV. EXPERIMENTAL SETUP AND RESULTS 
The Intel Real Sense D415 camera was used to capture 


images. Figure 8 shows the depth camera D415 [10]. 





Figure 7: The Intel Real Sense D415 camera with 
infrared module for depth sensing. 
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The camera takes the RGB image along with the depth 
information. The depth information was used to gauge 
the distance of the object from the frontal area of the 
user. So the direction in which they user is moving 1s 
constantly updated at a frame rate of 30 fps for a 
680x480 image resolution. In order to reduce high 
computations at real-time user feedback, 1080p 
resolution was used with 6 fps. 


The camera was embedded in a cap and placed on the 
head of a user. Figure 9 depicts the setup on top of the 
user’s head. Nylon straps were used to hold the cap in 





place and keep it stable. 


(b) 





(c) 





Figure 8: Sample 3-D depth image (a) RGB data (b) 
Figure 9: The depth camera implanted in a cap and associated depth data depicted as an image (c) depth 


placed on the head image after segmentation. 


Figure 8 shows a sample image and the associated depth Ste te ae ean alee ay.) eae anne 


information. Objects that are relatively near to the separated in 3 regions 1.e. the near objects (blue colour), 


camera are depicted in bluer color while farther objects gg ana eNO ORCC 1s (reer Or Our) and tar Objects 


are shaded as red (red colour). The three regions were selected so as to 
inform the user if there are any spaces among the nearest 


and the farthest objects. 


oa Intel RealSense Viewer ¥2.36.0 





Figure 9: The Intel Real Sense Viewer presenting the 
(a) RGB image as a 3-D object. 
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Figure 10 shows some case scenarios where the object 
(stool) was moved in various directions to present to the 
system different scenarios. It was noted that while the 
camera documentation suggests a nearest object 
detection at around Icm and a farthest object at 10 feet, 
the practical usage differed. The nearest object was 
detectable at 10cm and the farthest at 6 feet respectively. 





Figure 10: Effect on the depth image when object is 


brought near. 


V. CONCLUSIONS 
In this research work, the existing methods and systems 


developed for blind people assistance were discussed in 
detail. This included the analysis of various sensors 
used, complexity of each technique and its usability. A 
system based on depth camera in conjunction with 
image processing has been proposed. It utilized the 
depth information to inform the user about obstacles. In 
the future, it is aimed to include deep learning based 
object recognition so the user can be informed what 
object is present in the path. Further, the system can be 
embedded with AI to propose a path when obstacles 
appear. 
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